Improved Lite Audio-Visual Speech Enhancement

Numerous studies have investigated the effectiveness of audio-visual multimodal learning for speech enhancement (AVSE) tasks, seeking a solution that uses visual data as auxiliary and complementary input to reduce noise noisy signals. Recently, we proposed lite (LAVSE) algorithm car-driving scenario. Compared conventional AVSE systems, LAVSE requires less online computation some extent solves user privacy problem on facial data. In this study, extend improve its ability address three practical issues often encountered in implementing namely, additional cost processing data, asynchronization, low-quality The system is termed improved (iLAVSE), which convolutional recurrent neural network architecture core model. We evaluate iLAVSE Taiwan Mandarin with video dataset. Experimental results confirm compared can effectively overcome aforementioned performance. also suitable real-world scenarios, where high-quality sensors may not always be available.